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Abstract. Many complex systems are characterized by non-Boltzmann distribution functions of 
their statistical variables. If one wants to - justified or not - hold on to the maximum entropy 
principle for complex statistical systems (non-Boltzmann) we demonstrate how the corresponding 
entropy has to look like, given the form of the corresponding distribution functions. By two natural 
assumptions that (i) the maximum entropy principle should hold and that (ii) entropy should 
describe the correct thermodynamics of a system (which produces non-Boltzmann distributions) 
the existence of a class of fully consistent entropies can be deduced. Classical Boltzmann-Gibbs 
entropy is recovered as a special case for the observed distribution being the exponential, Tsallis 
entropy is the special case for q-exponential observations. 



Keywords: Generalized entropies, maximum entropy principle, non-Boltzmann distributions 
PACS: 05.20.-y, 89.75.-k, 05.70.-a, 05.90.+m 



It has been realized that many statistical systems in nature can not be described by 
a naive or straight forward application of Boltzmann-Gibbs statistical mechanics. In 
contrast to ergodic, separable, locally and weakly interacting systems, these systems 
are complex systems whose characteristic distributions often are of power-law type, 
or more complicated. Due to the existence of strong correlations between its elements 
complex systems often violate ergodicity and are prepared in states at the edge of chaos, 
i.e. they exhibit weak sensitivity to initial conditions. Further, complex systems are 
mostly not separable in the sense, that probabilities for finding a system in a given state 
factorize into single particle probabilities and as a consequence, renders these systems 
not treatable with Boltzmann single particle entropies [1]. In this context an adequate 
starting ground is provided by Gibbs entropies 



where T are the phase space variables, and B is the distribution function (Boltzmann 
factor) and [B] indicates functional dependency on B. Gibbs entropies can in principle 
incorporate arbitrary correlations via their explicit dependence on the Hamiltonian func- 
tion H(T) (potential term) describing the system. However, the Gibbs entropy in combi- 
nation with the usual Jaynes maximum ignorance principle (maximum entropy principle 
or variational principle) |2|] usually fixes the distribution function to be of exponential 
type B(H) ~ exp(— j5H), as demonstrated in detail below. For extending the concept 



INTRODUCTION 




(1) 



of statistical mechanics to complex systems, which are characterized by fundamentally 
different distribution functions, it becomes necessary to consider generalizations of the 
exponential distribution function. It is interesting to note that the exponential form of 
the distribution function is not a priori dictated by classical statistical mechanics, but in 
contrast much of classical statistical mechanics is built upon this special form of the dis- 
tribution function, as argued e.g. in |3|]. It is possible, e.g., to construct non-exponential 
distributions for particle systems. The form of these distributions depend on the form of 
the inter-particle potentials [0,[5l]. 

The aim here is to construct and deduce a correct entropy starting from a given exper- 
iment on an arbitrary statistical system jo]. Given a measured distribution function, (e.g. 
experimental data), what is the associated entropy which is compatible with the maxi- 
mum ignorance principle? This philosophy is very different from what has been done 
so far, i.e. take a (possibly modified) entropy and understand the resulting distribution 
functions. 



ENTROPIES FOR COMPLEX SYSTEMS ? 

In the following we ask whether one can construct a self-consistent theoretical frame- 
work where data, i.e. the measured distribution, serves as a starting point to construct 
an entropy which is consistent with both, the correct thermodynamic relations and the 
Jaynes maximum entropy principle D2D. According to this modification of logics it is 
sensible, in a first step, to modify or deform the log in Eq. (OQ) to a generalized loga- 
rithm A. The concept of deforming logarithms and thus modifying the form of entropy 
to accommodate a large body of experimental data from complex systems is not new 
flSBBEEIlElil. The generalized Gibbs entropy then reads 

S G [B} = -JdTB(H)A(B(H)) , (2) 

which, by a simple computation, performing the T integration over energy spheres, can 
be rewritten to 

S G [B) = -jde a(e) B (e) A(B (e)) , (3) 

where co(e) = J dT8(E — H) is the microcanonical multiplicity factor for the total 
system energy E. The associated maximum ignorance principle states that distribution 
functions of statistical systems are obtained by maximizing a functional G 



G = S G {B]-p j de(o(e)B(e)(e-U)-r(j de co (e) B(e) - l^j 



(4) 



provided the knowledge (measurement) of the expected energy value U alone. Here j3 
is the usual inverse temperature and y is the Lagrange parameter for normalizability. 
To keep close formal contact with usual statistical physics, we now represent the mea- 
sured distribution function by replacing the usual exponential function by some positive 
function £ ', i.e. 

exp{-P(E-U)-f)^£{-P(E-U)-f) , (5) 



where y is the normalization constant. £* is the deformed exponential function. It is the 
inverse function of A. The existence of the inverse function requires $, and thus A, to 
be monotonous. The minimum requirements for a distribution function B are that (i) 
B is positive and monotonic; (ii) B can be normalized, i.e. / d£ a>(e)B(e) = 1; (iii) B is 
sufficiently stable in time, such that it can be seen as a reasonable measurable probability. 
The generalized Gibbs entropy Eq. © then reads 

S G [B]= Jde<o(e)£(-P(e-U)-f)(fi(e-U) + f) , (6) 

and the usual definition of the expectation value 

(/) = f d£f{e)(D{e) -U)-y) (7) 

holds. Obviously the normalization constant y has to be chosen such that 

j d£(o{£) -U)-y) = \ . (8) 

A problem 

However, this approach raises a problem. Variation, i.e., dG = with respect to B 
obviously implies 

±BA(B) = -y-P(E-U) . (9) 

With the desired form of the distribution function, B(E) = (^(—^(E — U) — y) the only 
solution for the generalized Gibbs entropy Eq. © is the logarithm (!), i.e. A(B) log(5) . 
This is because Eq. © rewrites into A(B) +BA(B)' = -y-j8 (E-U). Inserting B(E) = 
£(—fi{E — U) — f) into A(B) further implies BA(B)' = f—y= const., which in turn 
implies, A(B)' = const. /B. Thus $ is forced to represent the usual exponential statistics. 
This is completely unsatisfactory for our philosophy! 



Solving the problem 

The above problem arises because of the extra term in Eq. ©, BA'(B), which is non- 
trivial for the A being anything other than the log. To cancel this term we suggest to 
further generalize the generalized logarithm A(B) to afunctional 

A(B)^A[B]=A(B)-r 1 [B] , (10) 

where [B] again indicates functional dependence on B. By substituting A by A in Eq. ©, 
we obtain the generalized- generalized entropy 



S GG [B}=S G [B} + 7][B} 



(11) 



where we have used that r\ is a constant with respect to e-integration and the 
normalization condition, Eq. ©. Now, the idea is that after variation with re- 
spect to B, the additional term j^rj [B] exactly cancels the problematic term, 
BA'(B), ore more precisely, —co(E)B(E)-^A(B(E)). The corresponding condition, 
3^tj [B] = G){E)B{E)j s K{B{E)), now dictates the form of 77 

r f B{e) 

r][B}= deco(e) dxA'(x)x+c , (12) 







up to an integration constant c. By partial integration of Eq. (PL?)) and substituting the 
result into Eq. (ITT]) we get for the generalized-generalized entropy 

f fB(e) 

S GG [B] = - J d£ co(e) J dxA(x)+c , (13) 

with c an integration constant which is only different from c, iff lim^o-^-Mjc) ^ 0. This 
constant c can be fixed to impose S GG = f° r the completely ordered state. Note that, 
based on a purely thermodynamic argument |14J], a very similar form of an entropy has 



been derived in H15N . In a very different context the same function was observed in Hi 611 . 

It can now easily be checked that this entropy, Eq. (TT3l . in combination with the 
standard maximum ignorance principle under the usual constraints, yields the measured 
distributions B. Define 

G = S GG [B}-fiJd£(Q(£)B(£)(£-U)-y(^jd£(D(£)B(£)-l^ , (14) 
and vary with respect to B, j^G = 0. Dividing by Q)(E) yields 

B(E)A'(B(E))-y-P(E-U)--^B(E)A(B(E))=0 , (15) 

or equivalently, A(B(E)) = —7 — j5(E — U). Using that $ is the inverse of A, the 
correct generalized distribution function, B(E) = ${— /3 (E — U) — 7), is recovered. The 
normalization constant y— 7 is equivalent with the Lagrange parameter of the maximum 
entropy functional. 



THERMODYNAMICS OF GENERALIZED-GENERALIZED 

ENTROPIES 

To show that the proposed entropy of Eq. (fT3l is fully consistent with the neces- 
sary thermodynamic relations we first differentiate Eq. (IT3T) with respect to U to get 
- / d£ CO(£)A(B(£))dB(£) /dU. Then, substituting -7-/3 (E - U) for A(B(E)) and par- 
tially integrating the result, taking into account that both, 7 and /3 depend on U, leads us 
to the (generic) result 

-Ls\B] = p . (16) 



The first and second laws for the generalized-generalized entropy are reviewed from 
II 1711 . where the entropy functional Eq. (fT3l) is written in its discrete form 

Sgg[B} = -Y [ B{E ' ] dxA(x)+c . (17) 

;J0 



The first law 

Since we use the usual concept of measurement, internal energy is given by U = 
L,- £,■#(£,■) . Let us vary U to get 

8U = J>Sf?(e;) +£fl| B ^Se ; , 

i i 

8Q -8W (18) 

because first 

8S GG = -8Y J f B(£ ' ) dxK{x)\ B= s = ^ £ i 5B = ^' 5 Q > < 19 ) 

i ^° i 

where Q denotes heat, and second, work is identified as usual as 

8W = -^B\ B=B 8ei . (20) 

The second law 

Let us write the relative entropy associated with the entropy of Eq. (fTTl) , which is of 
Bregman type, 

K K [B\\B]=Y [ B£ ' dx{A(x)-A(B)) . (21) 

Its variation with respect to B yields 

8K A [B\ \B] = £[A(x) -A(B)}8B( £i ) . (22) 

Now, take a maximum-entropy distribution, B = A _1 (7— /3(e,; — U)) as the reference 
state 

8K A [B\\B}=Y,Mx)8B(£ I )+PY,£i5B{£ l ) = -SS GG + l58Q . (23) 

i i 

Supposing the existence of a third state, B*, with 

K A [B*\\B}<K A [B\\B] , (24) 



one can write the variation as a superposition 



8K A [B\\B]=K A [XB* + (l-X)B\\B]-K A [B\\B] , VAe(0,l) . (25) 
Since relative entropy is convex, 

K A [XB* + (1-X)B\\B]-K A [B\\B}< XK A [B*\\B] + (1 - X)K A [B\ \B] (26) 

and 

8K A [B\\B] = X(K A [B*\\B]-K A [B\\B}) <0 , (27) 
this means that the Clausius inequalities hold for generalized-generalized entropies, i.e. 

8S GG >p8Q . (28) 

CLASSICAL EXAMPLES 

Let us give two examples to illustrate our philosophy, now switching back to continuous 
notation. Example 1: Classical Boltzmann distributions. If the experimentally measured 
distribution is of exponential type, B(E) ~ exp(— fiE), then A(B) ~ ln(5), and using Eq. 
(fT3l) yields the usual Boltzmann entropy, 

S GG [B} = - fde &>(e) B(e) ln(fi(e)) + l+c . (29) 



Example 2: Asymptotic power-law distributions. Let us suppose an experimental mea- 

i 

sured distribution function is a ^-exponential, i.e. B(E) = [1 — (1 — q)E] >-<?. Thus, the 
generalized logarithm is the so-called g-log, A(B) = ln q (B) = B il q 1 ■ Inserting as before 
gives the Tsallis entropy [Q, S] times a factor, 

S GG [B} = -—- fde (0(e) B(e) \n q (B(e)) + -^— + c , (30) 
2—q J 2—q 

where we require q < 2. The factor can in principle be absorbed into a transformation 
of /3 and y. At this point it is also obvious that in the case of power law distributions 
the question of normalizability can arise. Note however, that since p = coB has to be 
normalizable, an implicit regularization is provided by the maximal energy the 
observable system (represented by ft)) can assume. 



GENERALIZED DISTRIBUTIONS AND DUAL LOGARITHMS 



It has been recently noted [U, 10, 12, 8j] that the generalization of distribution functions 
immediately involves the existence of dual logarithms A*(x) = — A(l/x), as has first 
been defined in lfl8ll . For the generalized Gibbs entropy Eq. ©, for instance, one only 
has to consider the energy constraint (e) = U together with Eq. © to find S G [B] = f. 



For /3 = 0, this implies that B(E) = Z = const, because Z = J de (0(e). Therefore, 
S G [B] = -fde w(e)Z- 1 A(Z- 1 ) = -K{Z~ l ) = A*(Z). 



However, the dual logarithm property can be made genuine for arbitrary values of /3 
and generalized logarithms within the setting of our proposed generalized-generalized 
entropy Eq. (fT3l) . by observing that the partition function Z has to be defined in a 
deformed way, too. Using the definition of the deformed product x®y = <§ (A(x) + A(j)), 
as given in [3], the renormalization condition for the distribution function B can then be 
recast into the form 



which is to say, 7 = — A( 1/Z) or Z = \/£{—y). This becomes the defining equation for 
the generalized partition function Z. Inserting this into Eq. (fT3l) we finally get 



generalizing the famous formula S = log(Z) and giving us a deeper insight on how 
the functional 77 mediates between entropy and partition function. For self-dual logs, 
A* = A, which excludes the ^-logarithm (ln*(p) = hi2- ? (p)) and the Abe-log U, Eq. 
(1321) reduces to. S GG = J] + A(Z). 



We start by relaxing the restriction that distribution functions have to be exponentials 
and allow arbitrary types of distributions, B(E). By doing so we introduce correspond- 
ing generalized logarithms, A, the inverses of B, and suggest to construct the entropy 

Bis) 

of systems leading to non-exponential distributions, as S = — fde co(e) L dxA(x). 
By construction the observed distribution functions are compatible with the maximum 
entropy principle with the usual constraints. 

By demonstrating that this entropy is compatible with the correct thermodynamics, 
we claim that it makes sense to talk about the thermodynamics of complex statistical 
systems. This form of the entropy provides a potential tool that allows to carry out the 
usual thermodynamic operations in a fully self-consistent way. 

We have demonstrated that this entropy can be derived from the standard generalized 
Gibbs entropy (/ BAB) by adding a constant 77 which is functionally depending on the 
measured distribution function II 1911 . This r] is introduced to remove the problematic term 
BA'(B), arising from the variation of Gibbs entropy. For the Boltzmann-Gibbs-Jaynes 
case, A = log, and the problematic term is BA'(B) = 1. This 1 can be absorbed into the 
constant 7, and the problematic term has vanished. Similarly, for Tsallis distributions 
we have A = ]n q and BA'(B) = 1 + (1 — q)ln q . The 1 can get absorbed into 7, the 
(1 — q) ln ? into the <?-log term. In ^ a scale-and-shift Ansatz was suggested to fix 
BA'(B) = aA(B/b) + c, where again terms can get absorbed. From this perspective our 
suggestion is different: We re-define the form of entropy to remove the term BA'(B) 
altogether, not to absorb it. As a consequence we end up with the most general entropy 
which is compatible with the maximum entropy principle under the usual constraints 




(3D 



S GG = ri+A*(Z) 



(32) 



DISCUSSION 



allowing arbitrary distribution functions. Further, the usual thermodynamic properties 
have been shown to hold lfl7ll . 

7] allows for a physical interpretation of our philosophy: it somehow captures the 
numbers of states in phase space which depart from the classical Boltzmann case. This 
number may depend on long-range interactions or also parameters like temperature. 
The functional form of measured distributions, which is a kind of knowledge about the 
system, is thus naturally fed into the definition of the entropy of the (complex) statistical 
system. Effectively and formally, our result amounts to replacing the plnp term in the 
usual entropy by the integral, J A(p) . Obviously, classical Boltzmann-Gibbs entropy 
is obtained for specifying A(x) = ln(jc); Tsallis entropy is a further natural special 
case. Moreover, in this framework it can be easily shown that generalized distribution 
functions naturally imply the occurrence of dual logarithms. 

A further detail in our proposed entropy definition is that it does not contain any 
additional parameters, once the distribution is known. Once given the experimental 
distributions, there is no more freedom of choice of generalized logarithms, nor of the 
functional form of the constant r\ . 

S.T would like to thank Andrea Rapisarda for the kind invitation to Catania. 

REFERENCES 

1. L. Boltzmann, Sitzungsberichte der Kaiserlichen Akademie der Wissenschaften 66 (1872) 275. 

2. E.T. Jaynes, Phys. Rev. 106 (1957) 620. 

3. G. Kaniadakis, Phys. Rev. E 66 (2002) 056125. 

4. R. Hanel and S. Thurner, Physica A 351 (2005) 260. 

5. R. Hanel and S. Thurner, Physica A 365 (2006) 162. 

6. R. Hanel and S. Thurner, Physica A 380 (2007) 109 

7. C. Tsallis, J. Stat. Phys. 52 (1988) 479. 

8. C. Tsallis, M. Gell-Mann and Y. Sato, Proc. Natl. Acad. Sc. USA 102 (2005) 15377. 

9. S. Abe, Phys. Lett. A 224 (1997) 326. 

10. J. Naudts, Chaos, Solitons and Fractals 13 (2002) 445. 

11. T. Wada and A.M. Scarfone, Phys. Lett. A 335 (2005) 351. 

12. E.P Borges. lcond-mat/03045431 

13. T.M. Rocha Filho, A. Figueiredo and MA. Amato, Phys. Rev. Lett. 95 (2005) 190601. 

14. J.H. Weiner, Statistical mechanics of elasticity, John Wiley, New York (1983). 

15. M.P.Almeida, Physica A 300 (2001) 424. 

16. J. Naudts, Rev. Math. Phys. 16 (2004) 809. 

17. S. Abe and S. Thurner, arXiv:0708.1616, cond-mat.stat-mech 

18. J. Naudts, Physica A 316 (2002) 323. 

19. Generalization from a function to a functional was suggested by J. Lutsko, greatly simplifying 
notation (private communication). 



